The Generalized Spike Process, Sparsity, and Statistical Independence

We consider the best sparsifying basis (BSB) and the kurtosis maximizing basis (KMB) of a particularly simple stochastic process called the “generalized spike process”. The BSB is a basis for which a given set of realizations of a stochastic process can be represented most sparsely, whereas the KMB is an approximation to the least statistically-dependent basis (LSDB) for which the data representation has minimal statistical dependence. In each realization, the generalized spike process puts a single spike with amplitude sampled from the standard normal distribution at a random location in an otherwise zero vector of length n. We prove that both the BSB and the KMB select the standard basis, if we restrict our basis search to all possible orthonormal bases in Rn. If we extend our basis search to all possible volume-preserving invertible linear transformations, we prove the BSB exists and is again the standard basis, whereas the KMB does not exist. Thus, the KMB is rather sensitive to the orthonormality of the transformations, while the BSB seems insensitive. Our results provide new additional support for the preference of the BSB over the LSDB/KMB for data compression. We include an explicit computation of the BSB for Meyer’s discretized ramp process.